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Abstract — We present and study linear programming based 
detectors for two-dimensional intersymbol interference channels. 
Interesting instances of two-dimensional intersymbol interference 
channels are magnetic storage, optical storage and Wyner's 
cellular network model. 

We show that the optimal maximum a posteriori detection 
in such channels lends itself to a natural linear programming 
based sub-optimal detector. We call this the Pairwise linear 
program detector. Our experiments show that the Pairwise 
linear program detector performs poorly. We then propose two 
methods to strengthen our detector. These detectors are based 
on systematically enhancing the Pairwise linear program. The 
first one, the Block linear program detector adds higher order 
potential functions in an exhaustive manner, as constraints, to 
the Pairwise linear program detector. We show by experiments 
that the Block linear program detector has performance close 
to the optimal detector. We then develop another detector by 
adaptively adding frustrated cycles to the Pairwise linear program 
detector. Empirically, this detector also has performance close to 
the optimal one and turns out to be less complex then the Block 
linear program detector. 



L Introduction 

In this paper we consider detection of binary data in the 
presence of two-dimensional intersymbol interference (2D- 
ISI). Many important systems like magnetic and optical stor- 
age are modeled as 2D-ISI channel models. With an increasing 
demand for larger storage in smaller sizes, the ttaditional one- 
dimensional storage devices fall short. Thus there is a need 
for considering 2D storage devices. 

2D systems will naturally suffer from 2D ISI. One such 
2D storage system is the TwoDOS (two-dimensional optical 
storage) [1], [2]. Detection in TwoDOS reduces to detection on 
a 2D lattice or grid. A detailed survey of the 2D ISI detection 
(and coding techniques) is given in [3], [4]. 

It is known that the Vitterbi decoder achieves maximum 
likelihood sequence detection [5] for detection in ID ISI. For 
finite memory channels one can thus achieve optimal detection 
in ID ISI in linear time. In general, it is known that the 2D 
ISI detection problem (with additive Gaussian noise) is NP- 
complete [6]. As a consequence, there has been a lot of work 
in reducing the complexity of detectors. There has been a lot 
of work on developing low-complexity trellis-based detectors 
[7]-[12]. In [13]-[17] belief propagation (BP) based detectors 
are used for the 2D ISI channel. It was observed that the 



loopy BP detector performed poorly due to the presence of 
many small loops. Using a joint detection and coding (turbo 
equalization), loopy BP provided noise thresholds [15]. In [17] 
a generalized belief-propagation (GBP) channel detector is 
shown, experimentally, to have near-optimal bit-error-rate by 
considering regions of size 3x3. 

A. Our Contributions 

In this work we propose linear programming (LP) based 
channel detectors. As was observed in papers mentioned be- 
fore, the detection problem can be formulated as an inference 
problem on graphical models. We first formulate the natural LP 
based on the pairwise potentials of the factor graph. We show 
by experiments that (similar to loopy BP detector) this LP 
performs poorly. We then propose two methods to improve the 
detector based on enhancing the LP. The first one, the Block 
LP detector adds higher order potential functions in an exhaus- 
tive manner, as constraints, to the Pairwise LP detector The 
second detector identifies frustrated cycles (see Section IVT ). 
when the Pairwise LP produces a fractional solution, and 
adaptively adds them to the LP, which then enables us to 
recover the correct information word. We show empirically 
that the new detectors have a block-error performance close 
to the optimal one. Furthermore, the second detector turns out 
to be less complex than the Block LP detector 

II. Channel Model and Optimal Detection 

A. Channel Model: Uncoded Transmission 

We begin by describing the channel model. Consider an 
N X N grid. Let each point, 1 < i,j < N, on the 

grid represent an information bit taking value in {+1,-1}. 
We consider uncoded transmission. Thus the information word 
belongs is {+1, —1}^ . We denote by x the transmitted word 
and y as the received sequence. Both have length equal to N'^. 
The information bit is first observed through a 2-dimensional 
linear filter and then additive white Gaussian noise {J\f{0, a^)) 
is added to get the final noisy observation of the bit. More 
precisely, the 2D ISI channel model, we consider, is given by. 
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where ISI interaction strength is given by hij (strength less 
than 1) and d{k,l) denotes the neighborhood of {k,l). Also 
notice that the central bit, Xkj, has coefficient equal to 1 so 
that the bit under detection has the dominant contribution. 

B. Interference pattern 

We consider the nearest-neighbor interaction, specified by 
a 4-neighborhoo43. Furthermore, we will consider a periodic 
boundary. Let us illustrate the ISI interactions with an exam- 
ple. 

Example 1: Figure [T] shows a 5 x 5 square grid with circles 
denoting the bits. The bit at (3, 2) interacts with four of 
its neighbors at (2, 2), (4, 2), (3, 3), (3, 1). Also shown is the 
periodic nature of our interactions. The bit on the boundary, 
(5,4), interacts with (5, 5), (5, 3), (4, 4), (1, 4). Similar peri- 
odic interactions are also present (but not shown in the figure) 
for information bits which belong to the top-most row (they 
have one interaction with a bt on the bottom-most row). We 
consider periodic grid interaction so that we can rule out any 
boundary effects which would influence the LP detector 




Fig. 1. The figure on the left shows the bit at (3,2) interacting with its 4 
neighbors, (2, 2), (4, 2), (3, 3), (3, 1). Also shown is the interaction of a bit 
in the boundary. The figure on the right shows the factor graph for the 2D ISI 
detection problem with 9 bits and 36 potential functions denoted by squares. 
Each potential function is a pairwise interaction with strength between the 
nodes i and j given by RijXiXj. Although we do not shown them, to each 
there is a singleton potential function associated to each bit. 

In our experiments we will consider uniform ISI coefficients, 
i.e., hi J is same for all i,j to illustrate our methods. We further 
assume that the channel is perfectly known at the receiver In 
vector form the ISI channel can be written as y = H x + 
where iJ is the ISI matrix of all hij. Different _ff can model 
different applications. If the ISI coefficients are taken from 
a Gaussian distribution, then we can model Wyner's cellular 
network [17]. 

C. Optimal Detection and the Integer Program 

We denote by p{y\x) the transition pdf of the channel. Let us 
consider the optimal or maximum a posteriori (MAP) detection 
on the 2D ISI channel. We have 

X = argmax^g^_,_^-j_„2 exp(^ - ^(X! ^v^i^J ~ X! '^^^')) ' 

i>j i 

'Although it seems the hexagonal interaction is the design choice for the 
TwoDOS system because of its higher density [1] we perform our experiments 
on the 4-neighborhood for demonstrating our methods. 



where R = H, h = if'y (see [17] for details). 

Therefore the optimal detection problem reduces to solving 
the Integer program (IP), 

min RijXiXj -"S^ hiXi. (2) 

- 1- ' 1>J I 

Remark 2: The matrix R introduces next-to-neighbor inter- 
actions. Hence the above model is not planar Above we have 
replaced the notation {xij}(^,^j)(=[i.n]x[i.n] by {xi},e[i,N^]- 
The figure on the right in Figure [l] shows the factor graph of 

III. Main Results: LP Based Detectors 

An advantage of LP detectors over GBP detectors is that 
the LP provides a MAP certificate. More precisely, if the LP 
outputs an integer solution, then it must also be a solution 
to the IP and hence LP does MAP decoding in this case. 
However, in general, the IP is NP-hard and the output of the 
LP can be fractional. This implies that there is a gap in the 
LP approximation. In this situation the LP relaxation provides 
a lower bound (if we are considering the minimization of the 
objective function) to the value of the IP. 

We now provide an LP based on the pairwise potential 
functions and call this the Pairwise LP. This is analogous to 
applying loopy BP. 

A. Pairwise LP 

We can relax the above IP to 

mm^ X! RvjXiXjbij{xi,Xj) - X! X! ^i^i^ii^i) 

s.t. yi> j : ^ bij{xi,Xj) = 1, 

Vi > j \/xi,Xj : bi[xi) = y^bij{xi,x.j) 

Xj 
Xi 

< b^ixi) < 1, Vi, < bij{xi,Xj) < 1, 

Here bi{xi) and bij{xi,Xj) represent the beliefs of Xi and 
XiXj respectively. See [18] on how to derive the LP in terms 
of beliefs. 

B. Experiments with Pairwise LP 

Throughout the paper we will consider only low interference 
regime, i.e., hij are low. Consider a 9 x 9 grid and hij = 0.2 
uniformly for all 1 < i,j < N. We allow the noise, cr, to 
vary from 0.1 to 1.0 at an interval of 0.1. We run 2000 trials 
for each value of a. In each trial an information word x is 
picked u.a.r from {±1}^^ and is combined with a random 
noise configuration, w, to generate the observations y. Then 
y and _ff are fed to the Pairwise LP. If the output equals the 
transmitted information word, then we declare success, else 
there is an error We plot the word-error-rate (WER) versus 
SNR== 101ogio((4 • 0.22 ^ i)/cr2). 
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Figure |2] shows WER versus SNR. The WER is quite 
high and the Pairwise LP performs poorly. We observe that 
whenever the Pairwise LP fails, it is because the LP could not 
close the duality gap. I.e., the output of the Pairwise LP is 
fractional (and hence we declare an error). 
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Fig. 2. Figure shows simulation results for the performance of various LP 
detectors. We plot WER versus SNR for hi, j = 0.2 V 1 < i, j < and a = 
{0.1, 1.0}. The dashed curve depicts the perfomiance of the Pairwise LP. 
The Pairwise LP performs quite poorly. Also, whenever Pairwise LP fails, it is 
because the LP could not close the duality gap (LP gave fractional solution). 
The solid curve coiTesponds to Block LP. The Block LP peiforms much better, 
especially in the high SNR regime. For all simulations the Block LP output 
was integral, implying that Block LP did MAP decoding for this case. The 
curve in gray denotes the Pairwise LP-FC detector (see Section IV-A) . We 
observe that the Pairwise LP-FC curve sits right on the top of the Block LP 
curve and also gave an integer solution every trial. 

Remark 3: An important remark at this juncture is that 
when we solve the Pairwise LP (and any other LPs which 
will follow), we always add a very small random perturbation 
to the potential functions. This allows us to break ties when 
there are multiple integer solutions. 





Fig. 3. Figure on left shows a 5-clique. The figure on the right shows all the 
5-cliques with center on the second row. Boundary 5-cliques have the node 
at the opposite end present as the fifth node. 



Vi > j \fxi , Xj 



VC ViJeC: 

< b,{x,) < 1, 
< bcixc) < 1, 



bi{xi) -- 



- ^ ^ bij[xi^ Xj). 

Xi 

^ hc{xc) 

Xc\Xi,Xj 



Mi, < hij{xi,Xj) < 1, 
VC, C is a 5-clique. 



When we add all the 5-cliques to the LP, we have to 
make sure that they are consistent (marginalization condition) 
across any intersections with other cliques. It is not hard to 
see that any two 5-cliques intersect along an edge of the 
clique. Thus the intersections are pairwise cliques. Also, the 
5-cliques include the already present pairwise potentials as 
sub-cliques. Hence, we have the above consistency condition 
between a 5-clique and all of its constituent pairwise cliques. 
This relaxation resembles the GBP approach of [17]. 



IV. Improved LP Detectors 

From the above experiments it seems clear that Pairwise LP 
performs poorly. Most of the failure is because the LP outputs 
a fractional solution. In the high SNR regime we expect that 
MAP decoder should perform reasonably well. Hence, we now 
focus on improving the LP relaxation so that, at least in the 
high SNR regime, we recover the transmitted word. In other 
words, we aim to reduce the duality gap. 

A. Block Linear Program 

An immediate observation we make is that the pairwise 
interactions are not the most natural cliques present in the 
factor graph. It is not hard to see that the next-to-neighbor 
interactions (cf. Section Ill-Cb introduces a 5-clique as shown 
in Figure [3] Thus the first enhancement, is to add all such 
5-cliques to the LP. E.g., in a 9 x 9 grid, there are 81 such 
5-cliques which sit on each information bit. More precisely, 
we now have the following Block LP, 

mm^ ^ RijXiXjbij{x,,Xj) - ^^WiXibi{xi) 

~ i>j Xi.Xj i Xi 

s.t. yi > j : ^ bij{xt,Xj) = 1, 



B. Experiments with Blockwise Linear Program 

We consider the same setup as in Section ITlI-Bl but with the 
Block LP Figure |2] shows the WER versus the SNR (in dB) for 
both the Pairwise LP and Block LP. We see that the WER for 
the Block LP is much better than Pairwise LP for high SNR 
regime. In fact, in the high SNR regime every simulation trial 
was correctly solved by the Block LP. We also observe that 
for every simulation (i.e., for every SNR), the LP outputs an 
integral solution for each of the 2000 simulations. However, 
the output information word is not always the transmitted word 
(e.g., when WER is non-zero). We conclude that, for this case, 
the Block LP is doing MAP decoding. 

V. LP Detectors using Frustrated Subgraphs 

Although the Block LP served our purpose of providing an 
optima low complexity detector, it seems adding all the 5- 
cliques is unnecessary. In this section we investigate if there 
are '"smaller" optimal LPs. 

Our approach is to adaptively add constraints to the LP 
which, simultaneously, reduce the duality gap and are tractable 
(i.e., the number of such additional constraints are small and 
also each constraint involves only a small number of vari- 
ables). Such approaches, which try to get rid off the fractional 



4 



solution (or make the LP polytope tighter), have been used to 
improve the LP decoding of LDPC codes [19]-[21]. In [20], 
the LP is enhanced by eUminating the facet containing the 
fractional solution. In [19], [21], extra constraints are added 
by combining parity checks which correspond to violated 
constraints to improve the LP performance. Although our 
approach is in the same spirit, the main ideas have their origins 
in [22] and [23]. Similar ideas have been independently used 
in [24], [25]. Before we describe the basic idea let us first 
define the notion of a frustrated graph. 

Definition 4 (Frustrated Graph): Consider a constraint sat- 
isfaction problem (CSP) defined on n binary (boolean) vari- 
ables, X, and m check nodes. For each constraint node a there 
are only certain configurations of Xa which satisfy it. Then, 
we say that the graph is frustrated if and only if there is 
no assignment of x which satisfies all the constraint nodes 
simultaneously. ■ 

Let us now define a frustrated graph for our set-up. Assume 
that the output of Pairwise LP is a fractional solution, i.e., we 
have a duality gap. Consider all the potential functions (which 
have at least two variables) and their LP beliefs. E.g., consider 
one of the 5-cliques, say C, and its beliefs bc{xc)- We say 
that a configuration of xc satisfies C, if it has a non-zero 
belief, i.e., bc{xc) > 0. If the corresponding belief is zero, 
then we say that it does not satisfy C. In other words, the set of 
configurations which satisfy the potential function correspond 
to the support set of the belief. 

Lemma 5: If there exists a frustrated subgraph, then there 
is a duality gap. 

Proof: Indeed, suppose on the contrary there was no 
duality gap. This implies that the output of the LP is integer 
I.e., all the beliefs (on singleton potentials as well as higher 
order potentials) have only one configuration with belief 
equal to 1 (rest being equal to zero). Consider any subset 
of potential functions, C = {Ci, C2, . . . , C^}. Let a;^ denote 
the configuration such that bdix"^.) = 1.0. We claim that 
UiXp satisfies the CSP represented by C. This follows from 
the consistency imposed by the LP (between any higher order 
potential function and singleton potential functions). Thus no 
subgraph is frustrated. ■ 

Now if we add a frustrated subgraph as a constraint in our 
LP, then we ensure that this subgraph cannot be frustrated 
when we resolve the LP. In [22] it was found empirically that 
the random field ising model could typically be solved (duality 
gap closed) by adding frustrated cycles (cycles with odd 
number of frustrated interactions) arising in the LP solution. 
It is also known from Barahona's work (see references within 
[24]) that adding cycles is sufficient to solve the zero-field 
planar ising model. 

To ensure that the subgraph we add as a constraint to the 
LP becomes consistent (or is not frustrated), we need to add 
all its maximal cliques and their intersections to the LP. More 
precisely, we add the maximal cliques of the junction treeJl of 

- See [22] for a discussion on Junction trees. It can be sliown that running 
LP on the junction tree of a graph is optimal (equal to the IP). The complexity 
of the LP grows exponentially in the size of the maximal clique, which is the 
tree-width of the graph. Hence we focus on finding frustrated subgraphs of 
small tree-width which keeps the LP tractable. 



that subgraph as beliefs to the LP. 

The main challenge that remains is to find a frustrated 
subgraph (with low tree-width) in tractable time. In general, 
it is hard to find an arbitrary subgraph which is frustrated. 
As a result, we focus on finding frustrated cycles of the 
graph. This is a tractable problem and uses the implication 
graph method (to solve 2SAT problem) of [22], [26]. We 
describe it briefly here, for details see Appendix B in [22]. 
Consider all the two-projections of all the potential functions. 
I.e., for any bc{xc) consider all the bij{xi,Xj) V i,j £ C. 
In the implication graph each node i is present as i+ 
(for Xi — 0) and i_ (for Xi = 1). There is a directed 
edge present between i and j which represents the logical 
implication obtained from the potential bij{xi,Xj). To 
generate this logical implication, consider the set T of 
configurations of {xi,Xj) which render bij{xi,Xj) > and 
can introduce inconsistency. Thus, T is any of the following 
(01, 10), (01, 10, 11), (01, 10, 00), (00, 11), (00, 11, 10) and 
(00,11,01). Now one can draw the directed edges using 
this T. E.g., suppose that LP outputs beliefs such that 
fey (0,1) > 0, &,,(!, 0) > 0,6y(l,l) > 0,&,,,(0,0) = then 
T = (01,10,11) which would imply a directed edge from 
— > j_ and j+ — !> i_. Then a frustrated cycle is defined 
to be a directed cycle or path which visits both i+ and i_ 
for any i and one can find all such cycles and paths in linear 
time. 



A. Experiments using Frustrated Cycles 

The set-up is exactly same as previous two experiments. 
The detector, which we call it Pairwise LP-FC, is as follows. 



1) 


Run the Pairwise LP. Go to step 4). 


2) 


If the output is fractional, find all frustrated 




cycles (EC). Eor every ECs, add all the maxi- 




mal cliques of its Junction tree to the LP. This 




ensures that we only add triangles. 


3) 


Rerun the Pairwise LP. 


4) 


If output is integral, stop else go to 2). 



We observe in Eigure |2] that Pairwise LP-EC performs much 
better than the Pairwise LP and has the same performance as 
the Block LP. Furthermore, Pairwise LP-FC gave an integer 
output on every occasion. Thus in this case, Pairwise LP- 
FC does MAP decoding. We also remark that the number of 
triangles added is roughly 500 for each trial. This is much less 
than the total triangles present in the graph (= 85320). Also, 
the step 2) above is run only once, if it is required. 

B. Complexity of Block LP versus Pairwise LP with Cycles 

We measure the complexity of the LP by the number of 
nonzero entries in the LP constraint matrix. In Table |I] the 
Pairwise LP-FC entries correspond to the average (over 2000 
simulations) number of nonzero entries in the matrix. From the 
Table H] we see that, on an average, the Pairwise LP-FC has 
around half the number of nonzero entries in the matrix when 
compared to the same in Block LP. Thus, the Pairwise LP-FC 
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SNR 


Block LP 


Pai™ise LP-tC (avg) 


Pairwise LP-rC (max) 


20.6446 


4 X 10* 


1.0966 X 10* 


4.3578 X 10* 


14.6240 


4 X 10"* 


1.2163 X lO* 


4.2258 X 10* 


11.1022 


4 X lO'* 


1.4887 X 10* 


4.1202 X 10* 


8.6034 


4 X lO'* 


1.8523 X 10* 


4.1070 X 10* 


6.6652 


4 X 10*^ 


2.1440 X 10 


O.9ol8 X 10 


5.0816 


4 X 10-* 


2.2250 X 10* 


4.2258 X 10* 


3.7426 


4 X lO"' 


2.2162 X 10* 


3.6758 X 10* 


2.5828 


4 X 10'' 


2.1622 X 10* 


3.9618 X 10* 


1.5597 


4 X 10" 


2.0143 X 10* 


3.4558 X 10* 


0.6446 


4 X 10-* 


1.9363 X 10* 


3.5350 X 10* 



TABLE I 

Complexity comparison of Block LP and Pairwise LP-FC 



is a "smaller" (more sparse), on average, when compared to 
the Block LP, with the same performance. Also, the maximum 
nonzero entries (happens when step 2) is called) for Pairwise 
LP-FC is close to the Block LP one. 

VI. Discussion 

In this paper we develop channel detectors for the 2D ISI 
channel based on LP. Although the Pairwise LP performs 
poorly, both the Block LP and Pairwise LP-FC do MAP decod- 
ing. As we mentioned before, the advantage of LP detectors 
over GBP based detectors is that the LP detectors provide a 
MAP certificate. Another advantage is that one can formulate 
a systematic framework for improving the performance of LP 
detectors. As mentioned in [17], to date no systematic method 
of choosing regions (for the GBP algorithm) in a general 
graph exists in order to improve the performance. We end 
with possible open questions. 

(i) In this work we consider only the low interference regime 
of hi J = 0.2. It will be interesting to study the performance 
of the detectors when we vary the interference strength. 

(ii) An interesting research direction is to combine coding 
(or precoding) with channel detection and to develop a joint 
decoder and detector based on LP. 

(iii) Another open question is to study LP detectors when 
there is non-linear ISI [15]. This would introduce higher order 
interactions in the factor graph. 

(iv) LP is also used to decode LDPC codes when transmitting 
over binary-input memoryless channels [27]. An interesting 
question is to see if the LP decoder [18] enhanced using frus- 
trated cycles/subgraphs can lead to improvement in decoding 
thresholds. 
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